COMPARISON OF VOLUMES OF CONVEX BODIES 
IN REAL, COMPLEX, AND QUATERNIONIC SPACES 



BORIS RUBIN 

Abstract. The classical Busemann-Petty problem (1956) asks, 
whether origin-symmetric convex bodies in K." with smaller hy- 
perplane central sections necessarily have smaller volumes. It is 
known, that the answer is affirmative if n < 4 and negative if 
n > 4. The same question can be asked when volumes of hyper- 
plane sections are replaced by other comparison functions having 
geometric meaning. We give unified exposition of this circle of 
problems in real, complex, and quaternionic n-dimensional spaces. 
All cases are treated simultaneously. In particular, we show that 
the Busemann-Petty problem in the quaternionic n-dimensional 
space has an affirmative answer if and only if n = 2. The method 
relies on the properties of cosine transforms on the unit sphere. 
Possible generalizations are discussed. 



1. Introduction 

Real and complex affine and Euclidean spaces are traditional objects 
in integral geometry. Similar spaces can be built over more general al- 
gebras, in particular, over quaternions. The discovery of quaternions is 
attributed to W.R. Hamilton (1843) Q A variety of problems of differ- 
ential geometry in quaternionic and more general spaces over algebras 
were investigated by Rosenfel'd and his collaborators, in particular, in 
the Kasan' geometric school (Russia); see, e.g., [Rosj I VSS| IShij . Some 
problems of quaternionic integral geometry, mainly related to polytopes 
and invariant densities, were studied by Coxeter, Cuypers, and others; 
see [Cu, GNTl, GNT2] and references therein. 

In the present article we are focused on comparison problems for 
convex bodies in the general context of the space K", where K stands 
for the field R of real numbers, the field C of complex numbers, and the 
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skew field EI of real quaternions. Since H is not commutative, special 
consideration is needed in this case. 

Let, for instance, K and L be origin-symmetric convex bodies in 
with section functions 

SK{H) = voln-i{KnH) and SL{H) = voln-i{L f] H), 

H being a hyperplane passing through the origin. Suppose that Sk{H) < 
Sl{H) for all such H. Does it follow that voln{K) < voln{L)l Since 
the latter may not be true, another question arises: For which operator 
T> is the implication 

(1.1) VSk{H) < VSl{H) yn =^ voln{K) < voln{L) 

valid? Comparison problems of this kind attract considerable attention 
in the last decade, in particular, thanks to remarkable connections with 
harmonic analysis. The first question is known as the Busemann-Petty 
(BP) problem [BP] . Many authors contributed to its solution, e.g.. 
Ball [Baj, Barthe, Fradelizi, and Maurey [BFM] . Gardner [Gill 'Ga^l, 
IGKSj . Giannopoulos [Gi], Grinberg and Rivin |GRij . Hadwiger [HaJ, 
Koldobsky [K] , Larman and Rogers |LR] , Lutwak [Luj , Papadimitrakis 
[Pa] , Rubin |R5] , Zhang |Z2j . The answer is really striking. It is "Yes" 
if and only if n < 4; see |Ga3t IGKSt [Kf IKY] , and references therein. 

The second question, related to implication (11. II) . was asked by 
Koldobsky, Yaskin, and Yaskina |K YY] . It was called the modified 
Busemann-Petty problem. Both questions were studied by Koldobsky, 
Konig, Zymonopoulou |KKZj and Zymonopoulou |Zy| for convex bod- 
ies in C". The answer to the first question for is "Yes" if and only 
if n < 3. 

We suggest unified exposition of these problems for real, complex, 
and also quaternionic n-dimensional spaces and the relevant {n — 1)- 
dimensional subspaces H. All these cases are treated simultaneously. 
In particular, we show that the quaternionic BP problem has an affir- 
mative answer if and only if n = 2. 

The article is almost self-contained. Our proofs essentially differ from 
those in the aforementioned publications and rely on the properties of 
the generalized cosine transforms on the unit sphere |R2| IR3| IR7j . 

The setting of the quaternionic BP problem and its solution require 
careful preparation and new geometric concepts. The crux is that, un- 
like the fields of real and complex numbers, the algebra of quaternions 
is not commutative. This results in non-uniqueness of quaternionic 
analogues of such concepts as a vector space and its subspaces, a sym- 
metric convex body, a norm, etc. 
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Another motivation for our work is the lower dimensional Busemann- 
Petty problem (LDBP), which sounds like the usual BP problem, but 
the hyperplane sections are replaced by plane sections of fixed dimen- 
sion l<i<n — 1. In the case i = 2, n = 4, an affirmative answer to 
LDBP follows from the solution of the usual BP problem. For i > 3, a 
negative answer was first given by Bourgain and Zhang |BZj : see also 
[Kl IRZ] for alternative proofs. In the cases i = 2 and i = 3 for n > 4, 
the answer is generally unknown, however, if the body with smaller 
sections is a body of revolution, the answer is affirmative; see [GZj, 
[ZlJ . [RZJ. The paper [R8j contains a solution of the LDBP problem 
in the more general situation, when the body with smaller sections is 
invariant under rotations, preserving mutually orthogonal subspaces of 
dimensions i and n — i, respectively. The answer essentially depends 
on i. 

It is natural to ask, how invariance properties of bodies affect the 
corresponding LDBP prohleml 

Of course, this question is too vague, however, every specific example 
might be of interest. The article [KKZj on the BP problem in C" 
actually deals with the LDBP problem for (2n — 2)-dimensional sections 
of 2n-dimensional convex bodies, which are invariant under the block 
diagonal subgroup G of S0{2n) of the form 

G = {g = diag(5(i, . . . , fi-n) : Qi = ■ ■ ■ = Qn & S0{2)}. 

We will show that the BP problem in the n-dimensional left and right 
quaternionic spaces H" and H" is equivalent to the LDBP problem for 
{An — 4)-dimensional sections of 4n-dimensional convex bodies, which 
are invariant under a certain subgroup G C SO {An) of block diagonal 
matrices, having n equal 4x4 isoclinic (or Clifford) blocks. Every such 
block is a left (or right) matrix representation of a real quaternion and 
has the property of rotating all lines through the origin in by the 
same angle. We give complete solution to this "G-invariant" compar- 
ison problem in the general contest of dn-dimensional convex bodies, 

> 1, the symmetry of which is determined by complete system of or- 
thonormal tangent vector fields on the unit sphere S'^^^. The classical 
result of differential topology says, that such systems are available only 
on 5*^, 5*^, and S'''; see Section [231 We also study the corresponding 
modified BP, when the "derivatives" VSk and VSl are compared. 

Plan of the paper and main results. The significant part of the pa- 
per (Sections 2-4) deals with necessary preparations, the aim of which 
is to make the text accessible to broad audience of analysts and geome- 
ters. In Sections 2.1 and 2.2 we recall basic facts about quaternions 
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and vector spaces K", K G {M, C, H}. This information is scattered in 
the hterature; see, e.g., \KS\ ILout IPor|, [Tal IWol [Z] . We present it in the 
form, which is suitable for our purposes. Since EI is not commutative, 
we have to distinguish the left vector space H" and the right vector 
space H". 

In Section 2.3 we introduce a concept of equilibrated body in the 
general context of the space 21", where 2t is a real associative normed 
algebra. These bodies serve as a substitute for the class of origin- 
symmetric convex bodies in R". As in the real case (see, e.g., |Barj ). 
they are associated with norms on 2t". In the complex case, other names 
( "absolutely convex" or "balanced" ) are also in use |GLl IHoul IRob] . We 
could not find any description of this class of bodies in the quaternionic 
or more general contexts and present this topic in detail. 

In Section 2.4 we give precise setting of the comparison problem of 
the BP type for equilibrated convex bodies in K" e {M", C", H^', H;?} 
(see Problem A) and the corresponding problems B and C for G- 
invariant convex bodies in M.^ , N = dn. Here d = 1,2, and 4, which 
corresponds to the real, complex, and quaternionic cases, respectively. 
Section 2.5 contains necessary information about vector fields on unit 
spheres and extends problems B and C to the case d = 8. This value 
of d cannot be increased in the framework of the problems B and C. 

Section 3 provides the reader with necessary background from har- 
monic analysis related to analytic families of cosine transforms and in- 
tersection bodies. The latter were introduced by Lutwak |Lu] and gen- 
erahzed in different directions ; see, e.g, Gardner |Ga3j . Goodey, Lut- 
wak, and Weil [GLWj . Koldobsky [K|, Milman [Mi], Rubin and Zhang 
|RZj . Zhang |Zlj . Here we follow our previous papers |R2l IR3t [R7] . We 
draw attention to Section 3.2 devoted to homogeneous distributions 
and Riesz fractional derivatives D° = (— A)°/^, where A is the Laplace 
operator on M^. An important feature of these operators is that the 
corresponding Fourier multiplier |?/|" does not preserve the Schwartz 
space iS(M^) and the phrases like "in the sense of distributions" (cf. 
|KYY[ IKY[ |Zy] ) require careful explanation and justification. 

Section 4 is devoted to weighted section functions of origin-symmetric 
convex bodies. If K is such a body, these functions are defined as 
i-plane Radon transforms of the characteristic function Xk{x), (i.e. 
Xk{x) = 1 when x E K, and otherwise) with integration against the 
weighted Lebesgue measure with a power weight jxl'^. The usefulness 
of such functions was first noted in |R4] and mentioned in |RZt p. 
492]. Smoothness properties of these functions play a decisive role 
in establishing main results, and we study them in detail. Similar 
properties in the context of the modified BP problem in M" and C" 
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were briefly indicated in [KYYt IKYt |Zy| , however, the details (which 
are important and fairly nontrivial) were omitted. 

In Section 5 we obtain main results; see Theorems I5.4[ 15.51 15.81 and 
Corollaries 15.61 15. 7[ In particular, the Busemann-Petty problem in 
has an affirmative answer if and only if n < 2 + 2/d, where d = 1,2, 
and 4 in the real, complex, and quaternionic cases, respectively. 
Acknowledgement. I am grateful to Professors Ralph Howard, Daniel 
Sage, and Michael Shapiro for useful discussions. 
Notation. We denote by ct„_i = 27r"/^/r(n/2) the area of the unit 
sphere S*""^ in M"; SO{n) is the special orthogonal group. For 6&S"'~^ 
and 7 G SO{n), dO and (i7 denote the relevant probability measures; 
p^^n-i-j jg ^j^g space of C°°-functions on S"^"^ with standard topology; 
Pe(>S'"~^) is the subspace of even functions in ©(S*""^). 

In the following M„_a;(M) is the set of real matrices having n rows and 
k columns; M„(M) = M„^„(M); denotes the transpose of a matrix A\ 
/„eM„(R) is the identity matrix; K.fc = [F e M„,fc(M) : F^F = h} 
is the Stiefel manifold of orthonormal fc-frames in M"; Grk{V) is the 
Grassmann manifold of fc-dimensional linear subspaces of the vector 
space V. 

Given a certain class X of functions or bodies, we denote by the 
corresponding subclass of G-invariant objects. For example, C"~^(5'"~^) 
and D^{S"'~^) are the spaces of continuous and infinitely differentiable 
functions on S*""^, respectively, such that f{g9) = f{6) Wg E G, 9 E 
5"""^. An origin-symmetric (o.s.) star body in M", n > 2, is a compact 
set K with non-empty interior, such that tK G K \/t E [0,1], K = —K, 
and the radial function Pk{0) = sup{A > : \9 E K} is continuous on 
S"'~^. We denote by /C" the set of all o.s. star bodies in M". A body 
K eK.'' is said to be smooth if pK G Pel^""^)- 



2.1. Quaternions. We regard H as a normed algebra over M generated 
by the units Cq, ei, 62, 63 (the more familiar notation is 1, i, j, k, but 
we reserve these symbols for other purposes). Every element g G H is 
expressed as g = QqCq + qiCi + ^2^2 + §363 (g^ G M). We set 



The multiplicative structure in H is governed by the rules 
coCi = CiCo = Ci, 2 = 0,1,2,3, 
6162 = -6261 = 63, 6263 = -6362 = ei, 6361 = -6163 = 62, 



2. Preliminaries 



q = ?oeo - qiei - ^262 - ^363, 




2 _ 2 _ 2 _ _ 

Cl — ^2 — 63 — Cq. 
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The product of two quaternions p = poeo + piei + ^2^2 + PsCs and 
9 = + 9161 + q2^2 + ^363 is computed accordingly as 



(2.1) 



so that 



p<i = {poqo - PiQi - V2q2 - ^3^3)60 

+ (pogi + Piqo + P2qz - p^q2)ei 

+ (pog2 - Pigs + P2go + P3gi)e2 

+ {pm + Pi?2 - P2qi + P3go)e3, 



gg = gg=|g|, pq = qp, \pq\ = \p\\q\, q 
We identify 

M = {g G e : gi = g2 = g3 = 0}, C = {g G H : = gs = 0}, 

and denote by 5^(1) the group of quaternions of absolute value 1. 
There is a canonical bijection : EI — > M^, according to which, 



q = qoeo + qiCi + 5262 + 5363 

Sp{l) 

By (O, 

pq = (poqo + Piqi + ^2^2 + pm)eQ 



(?o,gi, 92,93)' 



-Poqi + Piqo - ^293 + ^392)61 



+ {-Poq2 + piqs + P2qo - P3qi)e2 + {-poqs - P192 + P2qi + Pzqo)e^, 



or 



(2.2) 
(2.3) 

Similarly, 
(2.4) 
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A'q = Ao = h, A[ 



10 
-10 
-1 
10 



(2.5) 



A' = 






-1 




1 

1 



-10 



A' — 
^3 — 



1 
0-10 
10 
-10 



One can readily see that Ai,A'^ e SO (A) and collections 

{AoVg, AlVg, A2Vq, AsVq}, {A'^Vq, A[Vq, A'^lhj, A'^Vq} 

form orthonormal bases of IR^ for every q e Sp{l) . This gives the 
following. 

Theorem 2.1. There exist "left rotations" Ai and "right rotations" 
A'^ (i = 1, 2, 3); such that for every a e , the frames 

{a, Aia, A2(T, Agcr}, {a, A'^a, A'^a, A'^a} 

form orthonormal bases ofW^. 

The left- and right-multiplication mappings p ^ qp and p — > in 
HI can be realized as linear transformations of R^, namely, 

(2.6) Vqp — LqVp, Vpq — RqVp, 



(2.7) Lq- 



Vqp 

Qo -Qi -?2 -^3 

qi qo -qs ?2 

?2 qs qo -qi 

q3 -q2 qi qo 



Rn 



qo -qi -q2 -qs 

qi qo qs -?2 

?2 -?3 qo qi 

qs q2 -qi qo 



These formulas define regular representations of H in the algebra M4 
of 4 X 4 real matrices: 

(2.8) pi : q ^ Lq, Pr ■ q Rq, 

so that pi{pq) = Pi{p)pi{q), Pr{pq) = Pr{p)Pr{q)- Clearly, 



(2.9) 

In particular, 
(2.10) A 



1=0 



i=0 



A'i — Rcii 



0,1,2,3. 



For any p, e H, matrices Lp and Rq commute, that is, 

(2.11) LpRq — RqLp. 
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Moreover, det(Lq) = det{Rq) = |g|^ (see, e.g., |Bel p. 28]). Since the 
columns of each of these matrices are mutually orthogonal, then, for 
|g| = 1, both matrices belong to SO {4). The map 



Spil)xSpil)^SOiA), ip,q) 



LpRqi 



is a group surjection with kernel {(eo,eo), (— eo,— eo)} [Port IWo ]. A 
direct computation shows that x-R qX — X'LqX — for every x & S'^ . It 
means that both Lg and Rq have the property of rotating all half-lines 
originating from O through the same angle cos~^go (such rotations are 
called isodinic or Clifford translations [Woj ) . We call Lg and Rg the 
left rotation and the right rotation, respectively. Note also that 



(2.12) 



JLgJ — Rg, 



JRgJ 



■'qi 



J 



-1 

/3 



It means that the left rotation becomes the right one if we change the 
direction of the first coordinate axis in M^. 
Similarly, if IK = C, we set 

C 9 c = a + z6 ^ v^ = {a,hY e M^ 



so that 

(2.13) = Vdc = McVd, c,deC, 



a —b 
b a 



Clearly, Mc G 5*0(2) if |c| = 1, and, conversely, every element of SO (2) 



has the form Mr 



cosip + 2 sm ip. 



2.2. The space K". Let K e {M,C,e}. Consider the set of "points" 
X = (xi, . . . , Xn), Xi G K, that can be regarded as an additive abelian 
group in a usual way. We want to equip this set with the structure 
of the inner product vector space over IK. The resulting space will be 
denoted by K". Unlike the cases IK = M and K = C, in the non- 
commutative case K = H it is necessary to distinguish two types of 
vector spaces, namely, right vector spaces and left vector spaces. 

We recall (see, e.g., [Art j ) that an additive abelian group X is a 
right M.-vector space if there is a map X x EI — > X, under which the 
image of each pair (x, g) G X x EI is denoted by xq, such that for all 
9; l\ q" ^ H and x, x', x" G X, 

(a) (x' + x")q = x'q + x"q; 

(b) x(g' + q") = xq' + xq"; 

(c) x{q'q") = {xq')q"; 

(d) xcq = X. 
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Similarly, an additive abelian group X is a left M-vector space if there is 
a map H x X — > X, under which the image of each pair {q,x) G EI x X 
is denoted by qx, such that for all q, q', q" G EI and x, x', x" G X, 

(a') q{x' + x") = qx' + qx"\ 

(b') (g' + (3'")x = g'x + 

(c') {q'q")x = q'{q"x); 

(d') CqX = X. 

According to these definitions, we define the left vector space EI^ 
to be the space of row vectors x = {xi,X2, ■ ■ ■ ,Xn), Xj G EI, with 
multiplication by scalars c G EI from the left-hand side (x — > ex = 
)). We equip EI" with the left inner product 

n 

(2.14) {x,y)i = J2^3yj- 

3=1 

The right vector space H" is defined as the space of column vectors 
X = (xi,X2, . . . ,Xn)'^, Xj G H, with multiplication by scalars c G EI 
from the right-hand side (x — > xc = (xic, X2C, . . . , x„c)"^) and with the 
right inner product 

n 

(2.15) {x,y)r = '^Xjyj. 

i=i 

Clearly, {x,y)i = {y,x)i, {x,y)r = {y,x)r- Furthermore, if x* = (x)"^, 
then 

(x, y)i = (x*, y*)r, (x, y)r = (x*, y*)i. 

If c is a real number, we can write cx = xc for both x G and 
X G EI". If K = C (or M) we regard C" (or M") as the space of column 
vectors and set 

n 

(2.16) {x,y) = '^Xjyj, 

i=i 

as in fl2.15p (in the commutative case, definitions fl2.14p and (12.151) 
coincide up to conjugation: {x,y)i = {x,y)r). 

Definition 2.2. We write K" for the vector spaces H", H", C", and 
M", equipped with the inner product defined above. 

There is a natural bijection h : K" R^, N = dn, where d = 
1, 2, and 4 in the real, complex, and quaternionic case, respectively. 



10 



BORIS RUBIN 



Specifically, 



(2.17) 3X = {Xi,...,Xn) 



[)4n 



(2.18) 



3 X-- 



Xi 












h ^ ^ 




2'n 







D4n 



(2.19) C" 9 x=(xi,...,xO 



n2n 



where Vx^ = h{xi). Abusing notation, we use the same letter h for both 
the scalar case, as in Section [^TT| and the vector case, as in fl2.17l) - fl2.19p . 
Formulas (12.61) and (I2.12p have obvious extensions. Namely, 

for X G 



(2.20) Vgx 

for X e (H") 

(2.21) Vx,= 



"^qxi 




LqVx^ 


"VqXn 




_ LqVx^ 



■-hqVx, L5 = diag(Lg, . . . ,Lg); 



^xiq 




Rq^xi 


"Vxnq 




. RqVx„ _ 



--TZgVx, 7^g = diag(i?, 



,g , . . . , Rq ) , 



(2.22) JLqJ = nq, JUqJ =hq, J = diag( J, . . . , J) . 

Matrices L^, TZq, and J have n blocks; hq and TZq belong to SO {An) 
and J'^ is the identity matrix. 

By (12. 2p . the inner product (I2.14p can be written as 

3 

(2.23) {x,y)i = ^{x,y)iei 



i=0 



(2.24) (x, y)i = Vx ■ AiVy, Ai = diag(Ai, . . .,Ai) {n blocks), 
Ai being defined by (12.30 . Similarly, by ([2? 



(2.25) 
(2.26) 



{x,y)r = ^{x,y)[ei, 

i=0 

(x, y)[ = Vx ■ A'iVy, A[ = diag(v4., . . . , A-). 
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By (I2inll and (Km . 

(2.27) JAiJ = A'i, ^ = 0,1, 2, 3. 

In the case K = C, for s G C" and c G C, owing to (12.131) . we have 

(2.28) Vc^ = v^c = McV^, Mc = diag(M„ . . . , M,) G S0{2n). 
Moreover, 

(2.29) {x, y) =v^-Vy- i{v^ ■ Bvy), 



(2.30) 



B = diae 





' -1 " 




' -1 " 


( 


1 


, . . . , 


1 



We introduce the following block diagonal subgroups consisting of n 
equal isoclinic blocks: 

(2.31) Gm,i = {g e SOiAn) : 

g = Lg = dia.g{Lg, . . . , Lg) for some g G H, |g| = 1}, 

(2.32) Gm,r = {ge S0{4n) : 

g = IZg = diag(i?q, . . . , Rg) for some g G H, |g| = 1}, 



(2.33) Gc = {ge S0(2n) : 

g = Mc = diag(Mc, . . . , Mc) for some c G C, |c| = 1}. 

If K = M, then the corresponding group Gr consists of two elements, 
namely, /„ and — /„. The groups Gm,i and Gn.r are conjugate to each 
other by involution J': 

(2.34) Ge,z = G^^rJ ■ 

Definition 2.3. We will use the unified notation G for groups Gh,;, 
GH,r; Gc, and Gr. 

2.3. Equilibrated convex bodies. It is known that origin-symmetric 
convex bodies in are in one-to-one correspondence with norms on 
M". What is a natural analogue of this class of bodies in spaces over 
more general fields or algehrasl Below we study this question in the 
general context of spaces over associative real normed algebras 21 with 
identity. Our consideration generalizes the known reasoning for real 
and complex numbers |Barl IGLj IHou[ IRob] . 

We assume that 2t contains real numbers and denote by |A| the norm 
of an element A in 21. Let V be a left (or right) module over 21. By 
relating vectors in V new elements, called points, one obtains an affine 
space over 2t |Rosj . We keep the same notation V for this affine space. 
As usual, a set A in y is called convex if x G A and y & A implies 
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ax + (3y G A for all a > 0, (3 > 0, a + f3 = 1. A compact convex set in 
V with non-empty interior is called a convex body. 

Definition 2.4. A set A in a left (right) space V over 2t is called 
equilibrated if for allxEA, \x&A (x\E A) whenever \E^,\\\<1. 

An equilibrated set in M" is just an origin-symmetric star-shaped 
set. The next definition agrees with standard terminology for normed 
algebras; cf. [Is, p. 655]. 

Definition 2.5. Let V be a left space over 21. A function p : V ^ M. 
is called a norm if the following conditions are satisfied: 

(a) p{x) > for all x e V ; p{x) = if and only if x = 0; 

(b) p{\x) = \X\p{x) for all X & V and all X G 21; 

(c) p{x + y) < p{x) + p{y) for all x,y E V. 

If V is a right space over 21, then (b) is replaced by 
(b') p{xX) = \X\p{x) for all X E V and all A G 21. 

Let \^ = 21" be the ra-dimensional left (right) affine space over 21. 
Every point x E V is represented as a; = Xi/i + . . . +Xnfn ( x = fiXi + 
• • • + fnXn), where Xi G 21 and /i = (1, 0, . . . , 0), . . . , /„ = (0, 0, . . . , 1) 

n 

is a standard basis in V . We set ||x||2 = (X] l^^iP)^''^- 

i=l 

Lemma 2.6. Let = 21" be a left (right) space over%. 

(i) If p : V is a norm, then 

(2.35) Ap = {xeV : p{x) < 1} 

is an equilibrated convex body. 

(ii) Conversely, if A is an equilibrated convex body in V , then 

(2.36) Pa{x) = \ \x\\a = inf{r > : x G rA} 
is a norm in V such that A = {x E V : ||a;||^ < 1}. 

The proof of this lemma is standard and is given in Appendix. 

In the following 21 = K G {M, C, H}; K" is any of the spaces 
R",C",H;^ or H;?; G G {G'R,Gc,Ge,/,G'e,r}; see Definitions O and 
12. 3t N = n, 2n, or An, respectively. Our next aim is to establish 
connection between equilibrated convex bodies in K" and G-invariant 
origin-symmetric star bodies in = h(K"'). We recall the notation 



(2.37) J = diag( 



' -1 " 




' -1 " 




/3 


, . . . , 


Is 


) 



{n blocks). 



Clearly, J' acts on ^ = (^i, ^2, • • • , ^4n) £ by converting ^1 into — ^1, 
^5 into —^5, and so on. 
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Theorem 2.7. Let A he a set in K" and let B = h{A) be its image in 
R^. Then 

(i) A is convex if and only if B is convex. 

(ii) A is equilibrated in if and only if B is G^^i-invariant and star- 
shaped. 

(iii) A is equilibrated in H" if and only if B is G^^r -invariant and star- 
shaped. 

(iv) A is equilibrated in C" if and only if B is Gc-invariant and star- 
shaped. 

(v) A is equilibrated in M" if and only if it is origin-symmetric and 
star-shaped. 

(vi) A set S in M'^" is star-shaped and Gu,i-invariant (or Gu,r -invariant) 
if and only if the reflected set JS is star-shaped and GM,r -invariant 
(GM,i-invariant, respectively). 

Proof, (i) Since h{ax-\-Py) =ah{x)-\-ph{y) ioY al\ a, P ^M. and x,y ^W, 
then A and B = h{A) are convex simultaneously. 

(h) Suppose that A C H" is equilibrated, ^ E B, and x = h^^{^). For 
any g G EI with |g| = 1 we have qx G A, and therefore, L^^ = h{qx) G 
B. Furthermore, for any A G [0, 1], = Xh{x) = h{Xx). Since Ax G A, 
then A^ G h{A) = B. Thus, B is Ge,/-invariant and star-shaped. 
Conversely, suppose that B = h{A) is star-shaped and GH,rinvariant. 
Choose any x E A, g G H, |g| < 1, and set q = Xu, X = \uj\ = 1. 
We have 

qx = Xlox = Xh~^h{ujx) = h~^[XL^h(x)]. 

Since B = h{A) is Ge.rinvariant, then Li^h{x) G B and since B is 
star-shaped, then XL^h{x) G B. Hence, qx = h^^[XLt^h{x)] G A. 

The proof of (iii) and (iv) follows the same lines with obvious changes. 
The statement (v) is trivial. The statement (vi) follows from fl2.22p . 
Indeed, let 5* be a star-shaped Ge ^-invariant set in R^" and let y G J'S. 
Then y = J'x, x E S, and for any g G EI with |g| = 1 we have TZgy = 
TlqJx = JJlZqJx = JhqX G JB, bccausc hqX G B. Furthermore, 
for any A G [0,1], Xy = XJx = JXx G JT'-B, because Ax G B. The 
reasoning in the opposite direction is similar. □ 

2.4. Central hyperplanes in K" and G-invariant Busemann- 
Petty problem in R^. Let = G K" : \\y\\2 = 1} be the 
unit sphere in K*^. Every hyperplane in K" passing through the origin 
has the form 

(2.38) ?/^ = {xGK": (x,i/)=0}, y E Sk-, 

where {x,y) is the relevant inner product; see (12.141) . (12.151) . (I2.16p . 
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If K = M, this is a usual (n — l)-dimensional subspace of M". If 
K = C, then, owing to fl2.29p . the equality (x, y) = is equivalent to a 
system of two equations 

^ ■ ^ = 0, ^-86 = 0, 

where ^ = h{x) G M^", 6 = h{y) G S*^"^^. This system can be replaced 
by one matrix equation 

(2.39) F2{ef^ = 0, F2{e) = [6, 86] G V2n,2, 

where V2n,2 is the Stiefel manifold of ortho normal 2- frames in M^". 
Equation (I2.39P defines a (2n — 2)-dimensional subspace of M^"". The 
collection of all such subspaces will be denoted by Gr2„_2(IR^")- 

In the non-commutative case IK = EI we have two option. If K" = H", 
then, owing to (12.231) . the equality (x, y)i = is equivalent to a system 
of four equations 

^■Ai9 = (^ = 0,1, 2, 3), 

or 

(2.40) F^,i{ef^ = 0, F^,i{e) = [AoO, AiO, A^, ^3^] e V^^a, 

where ^ = h{x) G M^", and 6 = h{y) G S^"-^^ (for simplicity, we use the 
same letters). If = H", then, by (12.251) . {x,y)r = is equivalent to 

(2.41) F^,r{ef^ = 0, F^,r{0) = [A'oO, A[e, A'^O, A'^O] G V^nA- 
Since A'^ = J A J (see (12:271) ). then 

(2.42) F^^r{0)=JF4,i{je) for every ^G^^"-^ 

Thus, (12.401) and (I2.4ip define two different {An — 4)-dimensional sub- 
spaces of M'*" generated by the same point e G 5^"-^ We denote 
by Gr^^^_4(R^"') and Gr^^^4(R^") respective collections of all such sub- 
spaces, which are isomorphic to S"^""^. By (12.421) . 

Given 6 G 5**^""^ {d = 1, 2, 4), we will be using the unified notation 
Hg for the {dn — (i)-dimensional subspace orthogonal to Fi{6) = 6, 
F2{9), ^4^/(6*), and ^4^,(6'), respectively. 

Proposition 2.8. The "right" manifold Gr^^'^4(M^'^) is invariant un- 
der the "left" rotations Lg, that is, 
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The "left" manifold Gr^^'_4(]R^'^) is invariant under the "right" rota- 
tions TZq, that is, 

Proof Let H G Gr4;'l4(M^"), that is, H is orthogonal to F4,r{0) = 
[Aq6, A[6, A^O, A'^6] for some 6 G 5"^""^ Since Lp and TZg commute 
for any p, g G H and = 7^g, (see fl2ll|) and flCTj) ). then LgA- = A'iLg 
and LqF4^r{0) = -^4,r(Lg6'). This imphes 

L,Gr^^_4(r") C Gr^:4(M^") 

for the corresponding bundles of subspaces. By the same reason, we 
have Fi^riO) = Lq-F4^r(L~^^) which gives the opposite embedding. The 
proof of equahty 7^gGr4„'_4(M4") = Gr4„'_4(M^") is similar. □ 

The above consideration enables us to give precise setting of the 
Busemann- Petty problem in and reformulate the latter as the equiv- 
alent lower dimensional problem for G-invariant convex bodies in M^. 
We recall that 

N = dn- n>l- d= 1,2,4:] G G {Gr, Gc, Ge.i, Gn.r}; 

see dMlD - fl2:33|) . We will be using the unified notation Grjv-d(K^) 
for the respective manifolds 

Gr„_i(R"), GrL_,(M2'^), Gt'^IM'"'). Gr^l4(M^-) 

of (A^ — (i)-dimensional subspaces introduced above. 

Problem A. Let A and B he equilibrated convex bodies in K"', n > 1, 

satisfying 

(2.43) voln^i{A n < voln-i{B n 

for all central IK-hyperplanes C,- Does it follow that voln{A) <voln{B)? 

Here volumes of geometric objects in K" are defined as usual volumes 
of their /i-images in R^, for example, 

volniA) = volN{hiA)), voln-i{A H = volN-d {hiA n 0)- 

The equivalent lower dimensional problem is formulated as follows. 
Problem B. Let K and L be G-invariant convex bodies in M.^ , with 
section functions 

SK{e) = volM-d{K n He), Sl^B) = voh-d{L n He), 
where He G Grjv-d(M^). Suppose that Sk{0) < SiiO) for all9 G S^-\ 
Does it follow that voIn{K) < voIn{L) ? 

We notice a fundamental difference between the usual LDBP prob- 
lem, where sections by all (A^ — d)- dimensional subspaces are compared. 
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and Problem B, where, in the cases d = 2 and 4, the essentially smaller 
(actually, (A^ — l)-dimensional) collection of subspaces comes into play. 

Since the question in Problem B may have a negative answer, we also 
consider the following more general problem, which is of independent 
interest . 

Problem C. For which operator V does the assumption VSxid) < 
VSl{9) we e S^-^ imply voIn{K) < voIn{L) ? 

2.5. Vector fields on spheres. Theorem [2lT] suggests intriguing links 
between possible generalizations of Problems B and C and the cele- 
brated vector field problem, which asks for the maximal number p{d) 
of orthonormal tangent vector fields on the unit sphere S"^'^ in M°'. 

We recall some facts; see [HestlHusllAdj . A continuous tangent vector 
field on S'^~^ is defined to be a continuous function V : 5*^^^ 
such that V((t) G a-^ for every a G 5'°'"^. If V(ct) = Act, where A 
is a d X (i matrix, the vector field V is called linear. Vector fields 
Vi, . . . , Vfc on S'^~^ are called orthonormal if for every a G S*"*^^, the 
corresponding vectors Vi(o"), . . . , Vfc(cr) form an orthonormal frame in 
W^. The following result is known as the Hurwitz-Radon-Eckmann 
theorem |Hut IRadl [E] ; see also [Og| . 




Theorem 2.9. Let d he a positive integer and write d = 2^^"'"''^, where 
t is an odd integer, s and r are integers with s > and < r < 4. 
Then the maximal number of orthonormal linear tangent vector fields 
on S'^~^ is equal to p{d) = 2'' + 8s — 1. 

The number p{d) is called the Radon-Hurwitz number. It is zero 
when d is odd. Adams [Adj extended this result to continuous vector 
fields. He proved that there are at most p{d) linearly independent 
continuous tangent vector fields on S'^^^. 

In the case p{d) = — 1, when there exist a complete orthonormal 
system of linear tangent vector fields {Vi, . . . , Vd-i} on 5'°'"^, the sphere 
S'^~^ is called parallelizable. The only parallelizable spheres are S^, S^, 
and 5'''; see Kervaire |Kej . Bott and Milnor |BMj . 

Complete systems of orthonormal linear tangent vector fields on 
S^, namely, {Aia, A2<7, As<j} and {A[a, A2a, A'^a}, where considered 
in Theorem 12. 1[ These produce a series of new examples, for instance. 



(2.44) {[7-Mi7]a, [-f-'A2-f]a, [7-^37]^^}, V7 G 0(4). 



A complete system of orthonormal tangent linear vector fields on S 
can be constructed, e.g., as follows. 
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Asa 
A^a 
A^a 
A^a 
A^a 



If a 




The corresponding matrices Ai, which are determined by permutation 
of indices of coordinates ai, . . . , ag and arrangements of ± signs, belong 
to 5*0(8). More systems can be constructed, e.g., as in fl2.44p . 

The following statement can be found in |Hes] in a slightly more 
general form. For the sake of completeness, we present it with proof. 

Lemma 2.10. 

(i) If a Ao is a linear tangent vector field on S'^^^ , then the dx d 
matrix A is skew symmetric, that is, A + = 0. 

(ii) // Aa = {Aia}'^zl is an orthonormal system of linear tangent 
vector fields on S'^^^, then 



AjAj + AjAi = for all l<i<j<d-l, 

AjAi = I for all l<i<d-l. 

Proof, (i) Let a ■ Aa = for all a G S'^^^. Equivalently, x ■ Ax = for 
all X G R''. Then, for all x,y e R'^, 



Lemma 2.11. Let Aa = {Aia}f^l be an orthonormal system of linear 
tangent vector fields on S"^~"^; Aq = I. Then 

d-l 



X ■ {A + A^)y = X ■ Ay + Ax ■ y 

= X ■ Ax + X ■ Ay + Ax ■ y + Ay ■ y = [x + y) ■ A{x + y) = 0. 

Hence, A + = 0. 

(ii) As above, for all x,y &R'^ we have 

X ■ {AjAj + AjAi)y = A,{x + y) ■ A,j{x + y) = 0, 
X ■ {AfA, -I)y=^ [A{x + y) ■ Ai{x + y) - {x + y) ■ {x + y)] = 0. 



This gives the result. 



□ 



(2.45) 




for every A = (Ai, . . . , A^) G S' 



'd-l 
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Proof. By Lemma [2. 101 

d-l d-l d-l 

g,{Afg,{A) = ( A.Af) ( A, A,) = X.X.AJA, = I. 

i=0 j=0 i,j=0 

Hence, c/a(A) EO{d). □ 

Some notation are in order. 

Definition 2.12. Let N = dn, d E {2,4,8}, n > 1. Given an or- 
thonormal system Aa = {Ajcrj^r/ of linear tangent vector fields on 
S"^'^, we denote 

(2.46) ^,(A) =diag((7,(A), ... ,(7,(A)) 

d-l d-l 

= diag ^ XiAi, . . . , XiAi^ (n equal blocks), 

i=0 i=0 

where A G S"^^^ . The corresponding class of block diagonal orthogonal 
transformations o/R^ (with n equal d x d diagonal blocks), generated 
by A, is defined by 

(2.47) G = G{n,d;A) = {geO{N) : g = gx{A) for some Ag^'^"^}. 
We also introduce N x N block diagonal matrices, containing n blocks: 

(2.48) A = diag(Ai,...,Ai) (z = 1,2,..., d-l), 

and set Aq = In- Given 6 G 5*^"^ we denote by Hg the {N — d)- 
dimensional subspace orthogonal to the d-frame 

(2.49) F,{e) = [6, A^e, Ad-ie] G VN,d 
and set 

(2.50) Gr^-d(M^) = {Hg : 9e 5^"^}. 

All objects in Definition 12.121 are familiar to us when c? = 2, 4 (see 
Section [23]). Thus, Problems B and C extend to the case d = 8. 

We recall that the set G of transformations and the set GrAr_d(]R^) 
of planes are determined by the orthonormal system A = {Ai}'^~l of 
vector fields, which is assumed to be fixed. 

The following lemma plays a crucial role in our consideration. 

Lemma 2.13. If H E Grjv-d(IR^), then every continuous G-invariant 
function f on S^~^ is constant on the {d — 1) -dimensional section 
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Proof. Let H = Hg he orthogonal to some (i- frame f l2.49p . Any point 
1] G 5*^^^ n is represented as 



d-l 



d-l 



V 



i=0 



i=0 



or T) = Q\{A.) 6] see (12 .46 p . In particular, if d = 4 and Ai have the form 
(12.31) . then ^a(A) is a block diagonal matrix with n equal blocks of the 
form 

Ao — Ai — A2 —A3 
Ai Ao —A3 A2 
A2 A3 Ao — Ai 
A3 — A2 Ai Ao 



i=Q 



Ly 



A = Aoeo + Aid + A2e2 + A3e3 G H; (cf. (I22D). 



Since gx{A) e G, then f{r,) = fiGxiA). 
result. 



f{9). This gives the 

□ 



3. Cosine transforms and intersection bodies 

It is known [R4l IR5t IR7t IRZ] that diverse Busemann-Petty type 
problems can be studied using analytic families of cosine transforms on 
the unit sphere. This approach is parallel, in a sense, to the Fourier 
transform method developed by Koldobsky and his collaborators [Kl 
IKYj . We shall see how these transforms can be applied to Problems 
A, B, and C stated above. 

3.1. Spherical Radon transforms and cosine transforms. We re- 
call some basic facts; see |R3t [R7j . Fix an integer i G {2, 3, . . . , — 1} 
and let Grj(]R^) be the Grassmann manifold of all i-dimensional linear 
subspaces ^ of M^. The spherical Radon transform, that integrates a 
function / G L^{S^^^) over {i — l)-dimensional sections S^~^ fl ^, is 
defined by 

(3.1) {R^fm= [ fiO)d^O, 

where d^9 denotes the probability measures on S*^"^ fl C,. The case 
z = A^ — 1 in (13.11) is known as the Minkowski- Funk transform 

(3.2) iMf)iu) = [ f{9) dj = (i?^_i/)(n^), u G 5^"^ 

J{e:e-u=o} 
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Transformation (13.11) can be regarded as a member (up to a multiplica- 
tive constant) of the analytic family of the generalized cosine transforms 

(3.3) (i?-/)(e) = 7;v.(«) / iFr^.er'-'-'' f{e)de, 

Here Pt^±6 stands for the orthogonal projection of 6 onto If / is 
smooth and Re a < 0, then R^f is understood as analytic continuation 
of integral (13.31) . so that 

(3.4) IjmR^f = = c. RJ, c. = 
In the case i = N — 1 we also set 



(3.5) {M^f){u) = {R%_J){u^)=^Nia) / /(^)|^ ■ ^1"-^^, 
/ X / X fTiv-ir((l-a)/2) 

(3.6) 7jv(«)= 27r(^-i)/2r(c^/2) ' ^ea>0, 1, 3, 5, . . . . 

Lemma 3.1. [R7l Lemma 3.2] Let a,(3 e C; a, /? ^ 1, 3, 5, . . . . // 

a + f3 = 2-Nandfe Ve{S^-^) then 

(3.7) M'^M^f = f. 

If a, 2 — N — a^l, 3,5, .. ., then M° is an automorphism of T>(.{S^~^). 

Corollary 3.2. T/ie Minkowski-Funk transform on the space T>f.[S^^^) 
can he inverted by the formula 

(3.8) (M)-^ = c^_i M'-\ c^_i = j;:,^^^, . 

Both statements amount to Semyanisty ^Se2j . who used the Fourier 
transform techniques. They can also be obtained as immediate conse- 
quence of the spherical harmonic decomposition of M°/. 

Lemma 3.3. |R7t Lemma 3.5] Let Re a > 0; a ^ 1,3,5,.... // 

f^L^{S^-^), then 

27r(»-l)/2 

(3.9) (i?.M"/)(0 = c(i?^+_VV)(e^), eGGr,(M^), c= , 

(3.10) (i?^_,M"/)(e^) = 

/// e Pel^^"^), i/ien / TO) and / TCTI) e:ciend to Rea<0 by analytic 
continuation. 
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Proof. We sketch the proof for the sake of completeness. For Re a > 0, 
(i?,M°/)(0 =77v(«) / d^u [ f{9)\9-ur-Ue. 

Since \9 ■ u\ = \Pt^9\\v0 ■ u\ for some vg G S^^^ fl ^, changing the order 
of integration, we obtain 

(i?,M"/)(0 = 7^(a) / f{9)\PH0r^d9 ! \ve-ur^d^u. 

The inner integral is independent of vg and can be easily evaluated. 
This gives (13.91) . Equality (13.101) is a reformulation of (13.91) . □ 

An origin-symmetric star body K in is completely determined 
by its radial function Pk{9) = sup{A > : G K}; see Notation. 
Passing to polar coordinates, we get 

(3.11) voUK n = ^ (^.Px)(0, e e Gr,(M^). 

The next statement follows from Lemma [2. 131 and plays the key role 
in the whole paper. 

Lemma 3.4. Let pk G D^{S^-^), N = dn; d G {1,2,4,8}, n > 1. 
Then for every subspace Hg G GrAr_d(M^) with 9 G S^^^, 

Af/2-d 

(3.12) vol^^,iKnHg)= (M'-'p^-'m. 

Proof. Applying successively (13. lip (with k = N — d), (13.41) , and (13.101) 
(with a = z + l — A^, i = N ~ d), we obtain 

vol^^,{KnHo) = ^^z±l{R^_,pl-^){Hg) 

= — N -d — ^^n-'^Pk ^^^^^ 

Since pk is G-invariant and M^^'^ commutes with orthogonal transfor- 
mations, then, by Lemma [2.131 M^^'^p^''^ = const on S^~^ fl if^ and 
(EH follows. □ 

Remark 3.5. In the classical case K = M, when N = n and d = 1, 
(I3.12P becomes a particular case of (13. lip : 

voL.^iKn9^) = ^{Mpl~^){9), 
n — 1 

where M is the Minkowski- Funk transform (13. 2p . 
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3.2. Homogeneous distributions and Riesz fractional deriva- 
tives. Given a G-invariant infinitely smooth body K in and a 
plane HoEGiM^diM-^) generated by 6* G S^^^, we denote 

(3.13) SK{e)=volN-d{Kf\He). 
Question: For which operator A", 

(3.14) A'^M^-'^p^^-'^ = {M^-"p^-'^){9) ? 

By (13.121) . an answer to this question would give us the corresponding 
equality for the section function 



(3.15) A-SKie) = ciM'-yr''m^ c , 

that paves the way to Problem C. By Lemma [3.11 we immediately get 

(3.16) = Mi-"Mi+°'-^. 

To make this explicit formula more transparent and convenient to han- 
dle, we extend our functions by homogeneity to the entire space 
and invoke powers of the Laplacian. This idea was formally used in 
|KYY| IKKZj . but it requires justification and some correction. Below 
we explain the essence of the matter. 

Let S{R^) be the Schwartz space of rapidly decreasing C°° functions, 
and 5'(M^) its dual. The Fourier transform of a distribution F in 
S'{R^) is defined by| 

(F,0) = (27r)^(F,0), Hy)= f me'^'-'^dx, </> G 5(M^). 



For / G Li(S^-i), let 

{E^f){x) = \x\^f{x/\x\), XGM^\{0}. 

This operator generates a meromorphic ^'-distribution, which is de- 
fined by analytic continuation (a.c.) as follows: 

oo 



{Exf,(j))=a.c. r^+^-\{r)dr, u{r)= f (6) (j){re) dO . 
Jo is'v-i 

The distribution Exf is regular if i?e A > — and admits simple poles 
at A = — A^, — — 1, . . .; see |GS] . If / is orthogonal to all spherical 
harmonics of degree j, then the derivative u^^\r) equals zero at r = 
and the pole at A = — — j is removable. In particular, if / is even, 
i.e., (/,yp) = (/,</._), ^.{9) = ^{-9) Vy. G V{S^-^), then the only 
possible poles of Exf are -A^, -A^ - 2, -A^ - 4, 



^Here and on, the notation (•, •) and (•,•) is used for distributions on and 
S"^"^, respectively. 
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Operator family {M"} (see (13 .Sp ) naturally arises thanks to the for- 
mula 

(3.17) [E.^N-af]'' = 2i""7r^/2 E„_iM"/, / G Pe(5^''), 

which amounts to Semyanistyi |Se2] . It holds pointwise for < -Re a < 1 
(see, e.g., Lemma 3.3 in [R2] ) and extends in the 5"-sense to all a G C 
satisfying 

(3.18) a ^ {1, 3, 5, . . .} U {1 - AT, -iV - 1, -AT - 3, . . .}. 

The Riesz fractional derivative D°'ip of order a G C of a Schwartz 
function ip is defined as a iS'(R^)-distribution by the rule 

(3.19) (27r)^(D>, 0) = 0), G 5(M^), 

where the right hand side is a meromorphic function of a with simple 
poles a = —N, —N — 2, . . .. One can formally regard D°' as a power 
of minus Laplacian, i.e., -D" = (— A)"/^. The case of negative Re a 
corresponds to Riesz potentials [St]. Since multiplication by \y\"' does 
not preserve the space iS(]R^), definition (13.191) is not extendable to 
arbitrary 5'(R''^)-distributions. 

To overcome this difficulty, Semyanistyi [SelJ came up with the bril- 
liant idea to introduce another class of distributions as follows. Let 

= \E'(]R^) be the subspace of iS(R^), consisting of functions u such 
that {d^uj){0) = for all multi-indices 7. We denote by $ = ^(R^) the 
Fourier image of \1', which is formed by Schwartz functions orthogonal 
to all polynomials. Let $' and \E'' be the duals of $ and \1', respec- 
tively. Two iS'-distributions, that coincide in the $'-sense, differ from 
each other by a polynomial. For any ^'-distribution g and any a G C, 
the Riesz fractional derivative D'^g is correctly defined by the formula 

(3.20) {D"g,u;) = (27r)-^(^, \yru;), a; G $. 

Clearly, multiplication by ||/|°' is a linear continuous operator on (but 
not on iS!); see |Rll ISKM] for details and generalizations. 

Lemma 3.6. Let a ^ {0, -2, -4, . . .} U {A^, + 2, + 4, . . .}. // 

/ G DeiS^-^), then 

(3.21) E_^M^-'^f = 2'^-'^D''~'^E_dM^~'^f 

in the ^' -sense. If, moreover, a — d = 2m, m = 0, 1, 2, . . and 

(3.22) {Drnfm = 2~'"^[{-ArE^,f]{x)U=e, 
then 

(3.23) (M'-'^fm = {DmM^-''f){e) 
pointwise for every 9 G S^~^ . 
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Proof. Replace a hj 1 — a and by 1 — d in (13.171) . Denoting Cq, = 

2-a^-jv/2 ^ 2-^7r-^/^ we get 

(in the 5'-sense). Using these formulas, for any test function G $ we 
obtain 

(E_,Mi-°/,^) = c^{[E^.Nf]\oo)=c^{E^^Nf,uj) 

= c^{Ed-Nf, br^'cl;) = c^{Ed-Nf, [D'^-'^u;]'') 

= c^{[E,.Mf]\ D^-^u) = c^c2\E.dM^~''f, D^-'^Lu) 

Let now a — d = 2m. Then _D"~'^ = (—A)™', and the same reasoning is 
applicable for any C°°-function supported in the neighborhood of the 
unit sphere. Hence, (I3.2ip holds pointwise in this specific case, and 
(E2S1) follows. □ 

Equalities (I3.12p and (13.231) imply the following 

Corollary 3.7. Let Sk{0), 9 G S^~^, he a section function Ii3.13\) 
of a G -invariant infinitely smooth body K in M^; = dn, n > 1, 
d G {1,2,4,8}. Let be a differential operator hS. 2^) . where 

2mj^N-d, N-d + 2, N-d + 4:,.... 

Then 

(3.24) {D^SkM = c{M'-'-'^p^~'m, c = 

3.3. Intersection bodies. We recall that /C^ denotes the set of all 
origin-symmetric star bodies in R^. According to Lutwak [Luj . a 
body K G K,^ is called an intersection body of a body L G K,^ if 
Pk{0) = volAr_i(L n 9^) for every 9 G S'^'^. A wider class of inter- 
section bodies, which is the closure of the Lutwak's class in the radial 
metric, was introduced by Goodey, Lutwak, and Weil |GLWj as a col- 
lection of bodies K G JC^ with the property px = Mp, where M is the 
Minkowski-Funk transform (13. 2p and p is an even nonnegative finite 
Borel measure on S^~^. The class of all such measures will be denoted 
byA^e+(5^-^). 

There exist several generalizations of the concept of intersection body 
[H IMII EH EH EH ■ One of them relies on the fact that the Minkowski- 
Funk transform M is a member of the analytic family of the cosine 
transforms. 
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Definition 3.8. [R7l Definition 5.1] ForO < X < N, a body K ^ K.^ is 
called a X-intersection body if there is a measure ^ G J^e+{S'^~^) such 
that = M^^^jj, {by Lemma{3Jl this is equivalent to M^+^~^p^. G 
Aie+{S^^^)) . We denote by the set of all such bodies. 

The equality = M^'^^fi means that for any ip G ©(S*^"^), 

JsN-i Js^~^ 

where for A > 1, {M^~''^ip){6) is understood in the sense of analytic 
continuation]!^ If A = A; is an integer, the class coincides with 
Koldobsky's class of fc-intersection bodies and agrees with his concept 
of isometric embedding of the space (M^, || ■ \ \k) into L_p, p = X [K\. 
In the framework of this concept, all bodies K G X^ can be regarded 
as "unit balls of A^-dimensional subspaces of L_a" . 

The following statement is a consequence of the trace theorem for 
cosine transforms; see [R71 Theorem 5.13]. 

Theorem 3.9. Let I < m < N , rj e Gr™(]R^), and let < X < m. If 

K eX^ in M^, then K Hr] eX^ in r]. 

This fact was used (without proof) in jKKZt Theorem 4]. In the 
case, when A = A; is an integer, it was established by Milman |Mij ; see 
|R7t Section 1.1] for the discussion of this statement. 

4. Weighted section functions 

Let K be an origin-symmetric convex body in M^. Given a point 
z G int{K) (the interior of K), we define the shifted radial function of 
K with respect to z, 

(4.1) p(2,f)=sup{A>0 : z + Xv G K}, {z,v) G n = int{K) x S^~\ 
which is a distance from z to the boundary of K in the direction v. 

Lemma 4.1. |RZt Lemma 3.1] If an origin- symmetric convex body K 
in has C"^ boundary dK , 1 < m < oo, then p{z,v) G C"^(f2). 

Proof. We recall the proof. Consider the function 

X — z 

V = g{z,x) = r, z G int{K), x G dK. 

\x — z\ 

Since dK is C™, g{z,x) is a function in int{K) x dK. When z 
is fixed, g{z,-) is a diffeomorphism from dK to S'^"^. By the 



^There is a typo in [R7| : In Definition 5.1 and in the subsequent equality on p. 
712 one should replace pK by p^. 
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implicit function theorem, x = f{z,v) is a function on Q. Thus, 
p{z, v) = \x — z\ = \ f{z, v) — z| is a C"^ function on Q. □ 

It was discovered by Gardner jGalj and Zhang |Z2] . that positive 
solution to the Busemann-Petty problem for convex bodies K in 
and is intimately connected with the volume of parallel hyperplane 
sections of those bodies; see also [Kl IKYj . This volume, which is a 
hyperplane Radon transform of the characteristic function Xk{x) of 
K, is represented as Anfiit) = voIn-i{K n{H + t9}), where t G M, 9 E 
S^~^, and is a hyperplane through the origin perpendicular to 6. 
It was noted in |R4] and in |RZl p. 492], that further progress can be 
achieved if we replace Au^eit) by the mean value of the i-plane Radon 
transform [Hel IR6] of some weighted function f{x) = \x\^xk{x). This 
mean value should be taken over all i-planes parallel to a fixed subspace 
^ G Grj(R^) at distance \t\ from the origin. Such averages for arbitrary 
/ (see ^R6| Definition 2.7]) play an important role in the theory of i- 
plane Radon transforms. Similar "weighted" section functions were 
later used in [KYYt Zy| . 

Let us proceed with precise definition. Given a convex body K G /C^, 
we define the weighted section function 

(4.2) A,,^(t,0 = / Af,{^ + tu)du, eeGr,(M^), t G M, 
where 

(4.3) Ao{C + tu) = / \xfdx, 

is the i-plane Radon transform mentioned above. Clearly, Ai^f^{t,^) is 
an even function of t. Let B = {x : \x\ < 1} be the unit ball in and 
let rA' = sup{t>0 : tBcK} be the radius of the inscribed ball in K. 

Lemma 4.2. If a convex body K G IC^ is infinitely smooth and [3 > m—i, 
then all derivatives 

A^lit,0= (^^J AAt,0, 0<j<m, 
are continuous in {—rx,rK) x Grj(M'^). 

Proof. Passing to polar coordinates in the plane ^ + tu, we get 

(4.4) A^(e + M= / al,{t)dv, 

pp{tu.v) 

a^^,it)= / ^ r^-'ir' + ty/'dr, 







COMPARISON OF VOLUMES 



27 



where p{tu,v) is the radial function (14.11) . It suffices to show that for 
P > m — i, all derivatives {d/dtya^^{t), j = 0, 1, . . . ,m, are continuous 
on {—rK,rK) uniformly in {u,v) G {S^~^ fl C,-^) x {S^~^ fl ^). Let, for 
short, pit) = p(tu,v). If m = and P > —i the uniform (in u and v) 
continuity of a^y(t) follows from Lemma 14.1 [ In the case m = 1 we 
have 

((iM)aJ,(t) = ai(t) + a2(t), 

where ai{t) = p'~^{p^ + t^)^/^dp/dt is nice and a2{t) = I3ta^^-^{t). If 
P > 2 — i we are done. Otherwise, ifl — i<P<2 — i, then 

(4.5) a2{t) = Pf+^-' / s'~\l + sy/^-' ds^O, as t 0, 

Jo 

and the result is still true. Continuing this process, we obtain the 
required result for all m. □ 

The next lemma is a slight generalization of the corresponding state- 
ments in [KYY] and Zy . 



Lemma 4.3. Let K he an infinitely smooth origin- symmetric convex 
body m M^, ^ G Gr,(M^), l<i<N. If-i<p<0, then Ai ^{t, < 
A,/3(0,O- // 2 - 2 < /3 < 0, then {d^ / dt^)A,^p{t,i)\t=^ < 0. 

Proof. Replace \x\^ in (14. 3 p by —P /q^^''^' z~^~^dz, /3 < 0, and change 
the order of integration. This gives 

+ tu) = -P z-^-^ voli{{Bi/, nK)n{^ + tu)) dz 
Jo 

where Bi/z is a ball of radius 1/z centered at the origin. The integral 
on the right hand side is well defined if —i < P < 0. Applying Brunn's 
theorem to the convex body Bi/^ fl K, we obtain 

vohiiBy, n K) n{^ + tu))< voliiiBy, H K) H ^ , 

which gives the ffist statement of the lemma. If2 — z</5<0, then, 
by Lemma the derivative {d'^/dt'^)Ai^i3{t,^) is continuous in the 
neighborhood of t = and the second statement of the lemma follows 
from the ffist one. In the case P = the result follows if we apply 
Brunn's theorem just to K. □ 



We recall some facts about analytic continuation (a.c.) of integrals 
1 



(4.6) 1(a) = -j— I t''~^f{t)dt, Rea>0. 



CO 

a-1 
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Lemma 4.4. Let m be a nonnegative integer, f G L-'^(R). 
(i) //, moreover, f is m times continuously differentiable in the neigh- 
borhood of t = 0, then I{a) extends analytically to Re a > —m. In 
particular, for — m < Re a < —m + 1, 



1 I'OO J.J 

(4.7) a.c. I{a) = —- / t^'^ [f{t) - -/(^■)(0) dt 

\ J JO j=0 



and 



(4.8) lim I{a) = (-l)'"/^'"^(0). 

(ii) // m is odd and f is an even function, which is m + 1 times con- 
tinuously differentiable in the neighborhood of t = 0, then Ij^T^ holds 
for —m — 1 < Re a < —m + 1 . 

Proof. All statements are well known [GSj . For instance, (ii) follows 
from the fact that all derivatives f^-'\t) of odd order are zero at t = 
and therefore, for m odd, the sum '^^Sq can be replaced by J2f=o- 
However, (14.81) is usually proved for functions, which have at least m + 1 
continuous derivatives at t = 0. We show that it suffices to have only 
m continuous derivatives. The latter is important in our consideration. 
Let 



t 

A-l 



t^ 



fitr]){l-r])^-Uv, A>0, 



r(A) 

be the Riemann-Liouville fractional integral of /. Note that 

m-l ■ 

and r'"(/'^/('"))(t) /(")(0)/m! as t ^ 0. Hence, for any e > 
there exists 6 = 6{e) > such that 

|t-™(J™/(™))(t) - /(™)(0)/m!| <e Vt G (0, 5). 



Setting a = tto — m, ao G (0, 1), we obtain 
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T{a) 



m—l 



1 



P 



j=0 



T{ao - m) Jq 
+/M(0) 

1 



tto r(ao — m) ml 



m— 1 



-m—l 



i=o -^^ 



dt = I1 + I2 + h 



If ao 0, then ao r(Q;o — m) m\ 



0, 



0. 



□ 



tto |r(ao — 
This gives the resuh. 

The next lemma estabhshes connection between weighted section 
functions, spherical Radon transforms, and cosine transforms. 

Lemma 4.5. Let ^ E Grj(R^), 1 < i < N. Suppose that 

aj^N-i, N-i + 2, N-i + A, 

and K is an infinitely smooth origin- symmetric convex body in M^. 
(i) If (3 > —i and Re a > 0, then 

(4.9) ^ 



r(a/2) 



P'-'A^pit.i) dt = c{RN-iM 



a+l+i-N a+f3+i\ / (-1. 



TC'/^ (TN-i-l 



(a + p + i) ajv-i r((iV - I - a)/2) ' 
(ii) If P > 1 — i, then ^.9^ extends to —1 < Rea < as 



(4.10) 



1 r°° 

—— / t^-'[AAt,O-AA0,O]dt 

i (a/^) Jo 



= c(/?;v-.M°+i+-^p^+'^+*)(e^). 

(iii) If P>2—i, then ^-10^ holds in the extended domain —2<Rea<0. 

(iv) If 13 > m — i and m > is even, then 

(4.11) ^%9r4^ (0,0 = ci (i?^-.Mi-+-^p^-™+^)(e^), 



2m+l 0f 
Cl 



TT*/^ (TN-i-l 



{(3-m + t) aN-i r((iV - I + m)/2) ' 
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Proof, (i) Consider the integral 

(4.12) g^^^(^) = ,^—[ \P^,xr+'-'' \xf dx, Rea>0, 

where denotes the orthogonal projection onto We transform 
(I4.12P in two different ways (a similar trick was used in |R5l p. 61] and 
[RZl p. 490]). On the one hand, integration over shces parallel to ^ 
gives 

9aAi) = ttAjtt, I \y\''^'-''dy [ \xfdx 

(4.13) = J,-— / r''AAt,Odt. 

r("/2) Jo 

On the other hand, passing to polar coordinates, we can express ga,/3 
as the generalized cosine transform (I3.3p . namely, 

9aA0 = r^-T^Tj^ri^J PK{ur^^^'\p^.ur'-''du 

{a + p + t)T{a/2) JsN-i 

= c^ARtpT'^-'m, 

27r(^-i)/2 



(a + p + i) (TN-i r((iV - I - a)/2) ■ 



Hence, by (13.9 



which gives (14.91) . 

(ii) By Lemma 14.21 (with m = 1) the derivative {d/ dt)AiAt-,C) is 
continuous in the neighborhood of t = 0. Keeping in mind that 

JVL = r((l-m)/2) 

and applying Lemma [4.4( i). we obtain (I4.10p . 

(iii) The validity of this statement for /? > 2 — i is a consequence of 
Lemma [4.21 (with m = 2) and Lemma [4.4( ii) (with m = 1). Consider 
the case j3 = 2 — i which is more subtle. Denote for short F{t) = 
^i./sitfO l^t first f3 > 1 — By Lemma [4.21 the derivative F'{t) is 
continuous in the neighborhood of t = 0. Since F is an even function, 
then -F'(O) = and the left hand side of fl4.1Up can be written as 

1 

(4.15) T^ri^ r-'Ait)dt, A(t) = F(t)-F(0)-tnO). 
r(a/2) Jo 
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By (1121) and g3D, 

A{t) = du I Au,v{t)dv 

where A,,,(t) = /(t) - /(O) - t/'(0), 

Jo 

To estimate A„^^(t), we write it as Au^^it) = Ii + 12, where 

rpitu,v) 

h= / r'-^[{r^ + ty^-r^]dr, 
Jo 

pp{tu,v) pp{0,v) 

l2= r'+^-^dr- r^+^-irfr-t[ai(0) + a2(0)], 

Jo Jo 

ai(t) = p'-\p^ + tY/^dp/dt, p = p{tu,v), a2(t) = Pta^;J{t). 

For /i, changing the order of integration, we have 

/i = ^ /"V-^r !\r^ + sY/^-Us = ^ f s^'^f'>'^-^h{s)ds, 
Jo Jo 4 Jq 

h{s) = r '\''^-\r, + lf'^-^d'n. 



Jo 

If j3 = 2 — i then h{s) = 0(log(l/s)) as s — > and therefore, Ii = 
0(tMog(l/t)) as t ^ 0. 

To estimate I2 we note that 02(0) = (see fl4.5l) ) and therefore, 

h = -^[p{tu,vy+^ - p{o,vy+^ -t{z + p)p{o,vy+('-'p\o,v)] 

Hence, h = 0{t^) as t ^ 0. Since all estimates above are uniform in u 
and V, then the function A{t) in fHTTHD is 0{t^ log(l/t)) as t 0. This 
enables us to extend this integral by analyticity to all Re a > —2. 
The statement (iv) follows from Lemma [4.21 (with m = 2) and (I4.8p . 

□ 



5. Comparison of volumes. Proofs of the main results 

We recall basic notation related to Problem B. Let K and L be origin- 
symmetric convex bodies in M^, N = dn, where n > 1, d & {1, 2, 4, 8}; 
G is the class f l2.47p of block diagonal orthogonal transformations of 
M^, which includes the groups Gk, Gc, Ge.^Ge.r; see (I2.3ip - (]2.33p . 
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The notation Gr7v_rf(]R^) is used for the respective manifolds f l2.50p of 
(A^ — (i)-dimensional subspaces Hg, 6 & S^~^, in particular, for 

Gr„_i(M"), GTl_,{R'n, Gr^-4(R'"), Gr^-4(K'"); 
see Section I2.4[ If K is an infinitely smooth G-invariant star body in 
M^, then, by Lemma [3.41 and Corollary 13.71 

(5.1) SK{e)=voi^.,{KnHg) = c{M'-''pf,''){e), 

(5.2) {D^SkM = c(M1-^-2>^-'^)(^), 
where 

c = 7r^/2-^arf_i/(iV - d), {D^fm = 2-'^[{-ArE^af]{x)U=e, 

(5.3) 2m^N-d,N-d + 2,N-d + 4:,.... 
Lemma 5.1. Let 

(5.4) a i {0, -2, -4, . . .} U {AT, AT + 2, AT + 4, . . .}. 

(i) Ij K and L are infinitely smooth G-invariant star bodies in such 
that (M"+i-^p^)(^) > and 

(5.5) (M^-V^-'^)(^) < (Mi-V^-'^)(e) e 
i/ien voIn{K) < voIn{L). 

(ii) // L zs an infinitely smooth G-invariant convex body with positive 
curvature such that {M"'~^^~^ p'l)[6) < for some e G S^-\ then there 
exists a G-invariant smooth convex body K for which ( [5. 5\) holds, but 
voIn{K) > voIn{L). 

Proof, (i) By Lemma [3. 

Nvoh{K) = ! pl{e)dd = (p^^pi) = (M^-V^-^M°+i-^pi). 

Since M"+^~^p^ > 0, we can continue: 

NvoIn{K) < (Mi->^-^M°+i-^p^) = (p^^pD- 

Now the result follows by Holder's inequality. 

(ii) Let ^{9) = (M°+i-^pi)(^) < for some 6 E S^-\ Then is 
negative on some open set Q C S^~^ and, by Lemma [3TTI p^ = M^~°'ip. 
Since (f is G-invariant, then (p < on the whole orbit GQ. Choose a 
function e V{S^-^) so that ^ 0, ij{e) > if 9 e GQ, and 
ip{6) =0 otherwise. Without loss of generality, we can assume ip to 
be G-invariant (otherwise, it can be replaced by ip{9) = J^ip{'y9) d'y). 
Define a smooth G-invariant body K by p^'"^ = p^'"^ - eM^+i-^V', 
e > 0. If e is small enough, then K is convex. This conclusion is a 
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consequence of Oliker's formula [Olj . according to which the Gaussian 
curvature of an origin-symmetric star body expresses through the first 
and second derivatives of the radial function. Applying M^~" to the 
preceding equality, we obtain 

which gives ( 15. 5p . On the other hand, 

(pi, Pl-" - Pk'") = e(Mi- V, M"+i-^^) = e{^, ^) < 

or {Pi, Pi~'^) < (Pl^Pk"'^)- Holder's inequality, the latter implies 
voIn{L) < voIn{K). □ 

Now, we investigate for which a the inequality (M""'"^~^p^)(^) > 
in Lemma 15.11 is available. 

Lemma 5.2. Let K and L he infinitely smooth G-invariant convex 
bodies in M^; = dn; n > 1; d E {1, 2, 4, 8}. Suppose that 

for some a satisfying 

(5.6) max(A^ - d-2,d) <a < N. 

Then voIn{K) < voIn{L). 

Proof. We apply Lemma [4.51 with ^ = Hq, i = N — d, and a replaced by 
a + d-N. By Lemma EH the expression (i?7v-iM"+^+*-^p°+'^+')(^^) 
in Lemma [4.51 transforms into Jq^^ = {M"'~^^~'^ p'^^){6) and the latter 
is represented as follows. 

• For a > N - d, (3 > d - N: 

„-l poo 
^ / ,a+d-N-l 



• For a = N - d, f3 > d - N: 

(5.8) I^,f, = ^Ar,.dAO,Hg). 

• For (a) N - d - 1 < a < N - d, 1 + d - N < f3 < 0, and 

(b) N - d - 2 < a < N - d, 2 + d-N<P<0: 

^^■^^ " Fiia + d-N)/2) 

POO 

X / t''+''-''-'[AN-dAt,He)-A^_,AO,He)]dt. 
Jo 
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• Fora = N-d-2, 2 + d-N<P<0: 

(5.10) I^,^ = -^-^A%_,^f,{0,He). 

Owing to Lemma 14.31 expressions fl5.7l) - fl5.10l) are nonnegative. Set 
P = d — a to get M"'^^~^ pj^ = Ia,d-a- Then combine inequalities in 
each case. We obtain the following bounds for a. 
For d = 1, N = n: max(r;, — 3, 1) < a < n. 
For = 2, 4, 8: 

(Kn holds if N -d<a < N. 
holds if a = N -d. 

holds if N -d-1 <a <N -d when N >2d+l; 

N-d-2<a<N-d when N > 2d + 2; 

d < a < N - d when 2d < N <2d + 2. 

(KTU\i holds if a = N -d-2, N>2d + 2. 
Combining these inequalities, we obtain fl5.6p . □ 

Remark 5.3. Operator M^~" = (M^+""^)~^ in Lemmas 15.11 and 15. 2^ 
that was originally defined by analytic continuation of the integral 
(13.51) . can be explicitly represented as an integro-differential operator 
P{6)M'^, where M"^ , 7 > 0, has the form (13.51) and P{S) is a polyno- 
mial of the Beltrami-Laplace operator 6 on S^~^; see |R2[ Section 2.2] 
for details. 

Lemma 15.21 leads to main results of the paper. The next statement 
gives a positive answer to Problem B. 

Theorem 5.4. Let K and L he G-invariant convex bodies in with 
section functions 

Sk{6)= volM-d{K n He), Sl(^) = vol^-diL n He), 
where He G Gr7v-(i(ffi^), N = dn, n> 1, G {1, 2, 4, 8}. Suppose that 

(5.11) Sk{9) < Sl{9) we G S^-\ 
Ifn<2 + 2/d, then voIn{K) < voIn{L). 

Proof. For infinitely smooth bodies the result is contained in Lemma 
15.21 (set a = d and make use of (15. ip ). Let us extend this result to 
arbitrary G-invariant convex bodies. Given a G-invariant convex body 
K, let 

K* = {x:\x- y\<lWyeK} 
be the polar body of K with support function 

hx'ix) = max{x ■ y : y E K*}. 
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Since hK*{-) coincides with Minkowski's functional || ■ H/r, then hK*{-) 
is G-invariant, and therefore, K* is G-invariant too. It is known |Schnt 
pp. 158-161], that any origin-symmetric convex body in can be ap- 
proximated by infinitely smooth convex bodies with positive curvature 
and the approximating operator commutes with rigid motions. Hence, 
there is a sequence {K*} of infinitely smooth G- invariant convex bodies 
with positive curvature such that h^* (0) converges to hx* {0) uniformly 

on 5*^"^. The latter means, that for the relevant sequence of infinitely 
smooth G-invariant convex bodies Kj = {K*)* we have 

lim max I 1 16*1 ■ — 1 16*1 lit I = 0. 

This implies convergence in the radial metric, i.e., 
(5.12) lim max |px,W -pxWI =0. 

Let us show that the sequence {Kj} in fl5.12p can be modified so that 
Kj C K. An idea of the argument was borrowed from ^RZj . Without 
loss of generality, assume that Pk{&) > 1- Choose Kj so that 

\PKm-pKm<-^ yees'^-' 

and set Kj = -j^Kj. Then, obviously, Pk'.{&) Pk{G) uniformly on 

S'^^^ as j oo, and 

PK' = -^PK, < -^{PK + -4t) - PK- 
J J J^l J + 1^ J + 1 

Hence, K'j C K. Now suppose that fl5.11l) is true. Then it is true when 
K is replaced by K'j, and, by the assumption of the lemma, vol at (-K'j ) < 
volAr(L). Passing to the limit as j — > oo, we obtain volAr(-ft') < volAr(L). 

□ 

The following theorem, which generalizes Theorem 4 from [KKZ] , 
shows that the restriction n <2 + 2/d'm Theorem 15.41 is sharp. 

Theorem 5.5. Let N = dn > 2d + 2, n > 1, de {1,2,4,8}. Then 
there exist G-invariant infinitely smooth convex bodies K and L in 
such that Sk{0) < Sl{0) for all 9 e S^'\ but voIn{K) > voIn{L). 

Proof. Let x = (xi, . . . , XnY ^ i = i^jA • • • ; ^j,d)^y 

L = {x : \\x\\i = (^^^ kjl"^) — 
i=i 

Clearly, L is a G-invariant infinitely smooth convex body. Let X be 
the {N — d + l)-dimensional subspace of , which consists of vectors 
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of the form (xi,i,X2, . . . ■,Xn)'^ ■ By [Kl, Theorems 4.19, 4.21], L fl X is 
not a A-intersection body in ]R^^'^+^ ifO<A<A^ — d — 2. Hence, by 
Theorem 13. 9[ L is not a A-intersection body for such A. It means (see 
Definition [31]) that {M^^^-^ p\){d) < for some 9 e S^-\ Set X = d 
to get dn > 2d + 2 and apply Lemma I5.1( ii) with a = d. This gives 
the result. □ 

Corollary 5.6. The Bus emann- Petty problem A in K", n > 1, has an 

affirmative answer if and only if n < 2 + 2/d. In particular, 
in M"'; if and only if n < 4; 
in C*; if and only if n < 3; 
in H" and H".- if and only if n = 2. 

Theorem 15.41 also implies the following. 

Corollary 5.7. Let d G {2,4,8}, i = N — d. The lower dimensional 
Bus emann- Petty problem for i-dimensional sections of N- dimensional 
G-invariant convex bodies has an affirmative answer in the following 
cases: 

(a) N = 4: {d = 2): i = 2, 
lb) N = 6 {d = 2): 1 = 4, 

(c) N = 8 (rf = 4) : i = 4, 

(d) N =10 (rf = 4) : i = 6, 

(e) N =16 (rf = 8) : z = 8. 

Another consequence of Lemma 15.21 which addresses Problem C, 
can be obtained if we set a = d + 2m in that Lemma and make use of 
Corollary 13.71 

Theorem 5.8. Let K and L be infinitely smooth G-invariant convex 
bodies in M^,- N = dn, n > 1, d G {1, 2, 4, 8}. Suppose that 

{-/\rE.dSK{e) < i-ArE^,SLi9) ye e s""-' 

for some m satisfying 

(5. 13) max( A - 2d - 2,0) <2m < N - d. 

Then voIn{K) < voIn{L). In particular, m can be chosen as follows: 

7-17 ( n — A n — 3 n — 2^ 
For d = l: m = if n<A, and m G < , , > if n > 4. 



^ 2 ' 2 ' 2 ^ 
For d = 2: m = if n < 3, and mG {ri — 3, n — 2} if n > 3. 
Ford = A: m = if n = 2, and mE{2n — 5, 2n — A, 2n — 3} if n > 2. 
Ford = 8: m = if n = 2, and 

mG{4n-9, 4n-8, 4n-7, 4n-6, 4n-5, } if n > 2. 
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6. Appendix: Proof of Lemma 12.61 

(i) We recall (see Definition [23]) that a function p : ^ M is a norm 
if the following conditions are satisfied: 

(a) p{x) > for all x & V; p{x) = if and only if a; = 0; 

(b) p{Xx) = \X\p{x) for all X e F and all A G 21; 

(c) p{x + y) < p{x) + p{y) for all x,y E V. 

If V" is a right space over 21, then (b) is replaced by 

(b') p{xX) = \X\p{x) for all X eV and all A G 21. 

Let V he a left space (for the right space the argument follows the 
same lines with (b) replaced by (b')). Suppose that p : V ^ M. is a. 
norm and show that 

(6.1) Ap = {x eV -.pix) <!} 

is an equilibrated convex body. Let x,y & Ap. Then for any nonnega- 
tive a and /5 satisfying a + (3 = 1, owing to (b) and (c), we have 

p{ax + Py) < p{ax) + p{l3y) = ap{x) + l3p{y) < a + (3 = 1. 

Hence, ax + Py E Ap, that is, Ap is convex. Since for every A G 21 with 
|A| < 1, (b) implies p{Xx) = \X\p{x) < 1, then Ax G Ap. Thus Ap is 
equilibrated. To prove that Ap is a body, it suffices to show that Ap 
is compact and the origin is an interior point of Ap. To this end, we 
first prove that p is a continuous function. Let x = Xifi + . . . Xnfn, as 
above. By (b) and (c), 

p{x) < p{xifi) + . . .+p{Xnfn) = \Xl\p{fl) + . . . + \Xn\p{fn) 

n 

^ tY] kjl, 7 = max p{fj). 

Now for X = (xi, . . . , x„) and y = {yi, . . . , y^), owing to (c), we have 
p{x) < piy) + p{x — y), piy) < p{x) + p{y — x) = p{x) + p{x — y). 
Hence, 

n 

\p{x) - p{y) \ < p{x - y) < -f^\xj ~ yj\, 

i=i 

and the continuity of p follows. Furthermore, since p{x) > for every x 
on the unit sphere Q = {x E V : ||x||2 = 1} and since p is continuous, 
there exists S > such that p{x) > 6 for all x G fi. If x G 
and x' = x/||x||2 G Q, then 1 > p{x) = ||x||2p(x') > 5||x||2, i.e., 
||x||2 < S^^. Thus, Ap is bounded. Since Ap is also closed as the 
inverse image of the closed set < A < 1, it is compact. 

To prove that Ap is a body, it remains to show that Ap contains the 
origin in its interior. Since p is continuous and Q is compact, there is 
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a number f3 > such that p{x') < j3 for all x' G VL. Then the open 
ball Biii3 — {x & V : \\x\\2 < 1 / 13} lies in Ap, because for x e 

p{x) = \ \x\\2p{x') < ||x||2/3 < 1. 

(ii) Suppose that A C V is an equilibrated convex body and let us 
prove (a)-(c) for pa{x) = inf{r > : x G rA}. Since A is equilibrated, 
then G ^ and therefore, Pyi(O) = inf{r > : G rA} = 0. Con- 
versely, \ipA{x) = mf{r > : X G rA} = 0, then for every /c G N, there 
exists Tfc < 1/k such that x G r^A. Since A is equilibrated, then r^A is 
equilibrated too, thanks to the following implications that hold for all 
A G K, |A| < 1: 

X G rkA =^ — e A =^ — G A =^ Ax G rjt^. 

Since TfeA is equilibrated, then G rkA for all k. Passing in a; G rkA 
to the limit as /c — > 00, we get x — 0. This gives (a). 

Let us check (b). For A = 0, (b) follows from (a). Let A 7^ 0. Since 
A is equilibrated, then for every r > 0, Xx & rA if and only if x G j^^. 
Hence, 

r 

Pa{Xx) = inf{r > : Ax G rA} = inf{r > : x G ttt^} 

II 

= |A| inf{r > : x G rA} = \X\pAix). 
To prove (c), choose a, /3 > and let x G a A, y G f3A. Then 

x + y= a + — — - + ^-^ 

\a + p a a + p p 

Since the points a~^x and P^^y are in A and A is convex, the weighted 
sum in parentheses is also in A, and therefore, x + y e {a + I3)A. This 
gives pa{x + y) < a + p. By letting a ^ pa{x), /3 = PA(y), we arc 
done. □ 
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